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Abstract 

This study investigated the performance of graduate students enrolled in introductory statistics 
courses. The course in Fall 2005 was delivered in a traditional face-to-face manner and the same 
course in Fall 2006 was blended by using an online commercial tutoring system (ALEKS) and 
making attendance of several face-to-face classes optional. There was no significant difference 
in the /-test comparing performance in the courses, which used the students’ combined score on 
two mid-terms and the final exam to indicate performance. The ANCOVA analyzing influences 
on performance in the blended class yielded no significant influence for gender, ethnicity, age, or 
class type (traditional vs. blended), but a significant influence from students’ incoming GRE- 
Quantitative score. Seven Likert questions on students’ perception of blended learning were not 
correlated with student performance. Three focus groups - comprised of low-, medium-, and 
high-performing students - revealed three themes and several subthemes and differences based 
on students’ performance level. 
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Introduction 

There is perhaps no more difficult subject to teach in a college of education than statistics. The 
types of students who are drawn to education are often motivated by a desire to work with 
students, improve schools, and pursue a career in a range of service occupations. Individuals 
drawn to graduate study in education are often intelligent and dedicated, but not all have a 
background in mathematical studies. For the teacher of statistics, some of these students may 
actually exhibit a high level of fear and aversion to the subject, which complicates the learning 
process. The challenge for any instructor of statistics is to prepare graduate students to 
understand and critique research studies using a range of statistics and to confidently design a 
high-quality research study of their own. 

This study was motivated by the desire of a statistics professor to test a blended learning 
approach to teaching statistics. Statistical Methods Applied to Education I - or Stat I - covers 
the use and interpretation of such topics as frequency distributions, central tendency, Pearson 
correlation, linear regression, introduction to probability, normal distribution, hypothesis testing 
and use of computer software in statistical analysis. It is a required course for the 15 master’s and 
doctoral programs in this particular college of education. Therefore, it is taught every tenn and 
must address the learning needs of a diverse group of students. 

For Fall 2006, Stat I was augmented by a web-based software system called Assessment and 
LEaming in Knowledge Spaces or ALEKS (http://www.aleks.com) . The system is an artificial 
intelligence-based learning environment that provides assessment of learning, remediates gaps in 
student learning and preparation, monitors student performance, and provides explanations and 
feedback. Traditional pedagogical approaches were combined with ALEKS to provide students 
with a blended class: part online and part face-to-face. Using a mixed-methods study design, this 
research evaluates how well the approach worked in comparison to a more traditionally taught 
statistics course, analyzes in depth the influence of blended learning on student performance, and 
discusses improvements to be made in future statistics courses. 
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Theoretical Background 


Blended Learning 

The Sloan-C Consortium adopted the following definition to provide some precision to 
the term of “blended learning:” 

1. Courses that integrate online with traditional face-to-face class activities in a planned, 
pedagogically valuable manner; and 

2. Where a portion (institutionally defined) of face-to-face time is replaced by online 
activity (Picciano, 2006). 

To date, several books and articles on blended learning have focused on how to and when to 
blend (Bonk & Graham, 2005; Kaleta, Skibba, & Joosten, 2007; Thorne, 2003; Vaughan, 2007). 
Studies on blended learning have only recently begun to be undertaken by many researchers. 
The Sloan-C Consortium has recently published a book on blended learning (Picciano & 
Dziuban, 2007) that focuses on theories and questions on blended learning. Currently, research 
on blended learning has focused on effectiveness (Vignare, 2007), with some studies finding 
positive results (Dziuban, Hartman, Moskal, Sorg & Truman, 2004; Graham & Robison, 2007; 
McCombs & Vakili, 2005) and others questioning whether blended learning is really better than 
fully online learning (Reasons, Valadares & Slavkin, 2005; Wu & Hiltz, 2004). It does not 
appear that many studies have been done to understand the specific mechanisms by which 
blended teaching impacts learning. In a comparison of a traditional versus blended class in 
accounting, Chen and Jones (2007) found that while both methods produced similar learning 
outcomes, the students in the blended class felt their analytical skills improved more than 
students in the traditional class, but traditional class students felt the clarity of instruction was 
higher. With inconsistent findings such as these, more research is certainly needed to explore 
what types of learning may be more effective in which setting - face-to-face or online - or how 
to best combine or integrate the two settings to ensure student learning. 

Media Comparison Studies 

Media comparison studies comprise the majority of the studies included in the “no significant 
difference” literature (http://www.nosignificantdifference.org) . Russell (1999) prepared several 
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editions of this book which compiled “no significant difference” studies, now totaling 355 
studies and growing. These studies often compare end-of-course grades or final exam scores of 
two courses, one taught in a traditional fashion and the other taught via some technology, either 
satellite, interactive television, etc. Although it is difficult to conduct experimental studies in 
educational research, these studies have been criticized as being too broad and poorly designed, 
rarely matching students or carefully controlling treatments, assuming that using the same 
instructor to teach both courses would be sufficient. It is argued that, in such a design, too many 
other variables can intercede to affect the outcome, making such comparisons unable to 
discriminate the actual influence - if any - of the technology or delivery method. 

Despite this criticism, media comparison studies continue to appear in the literature and are 
undertaken by faculty. Meyer (2004) has argued that media comparison studies serve a useful 
purpose since the individual instructor has a need for proof that the technology is not harmful 
and perhaps may even be beneficial for student learning. This was termed “personal journey 
research,” or research undertaken not for its ability to contribute to the literature or stretch the 
boundaries of what is known, but for its ability to help individual instructors come to better 
understand the technology and prove to themselves that it helps students learn. Therefore, it is 
unlikely that media comparison studies will vanish anytime soon. 

In this study, the initial question is to compare the performance of students in the blended 
learning class using ALEKS with the performance of students in an earlier class not using 
ALEKS. This was an essential first step for the instructor. Second, the major focus of this study 
was to collect and analyze students’ perceptions of the blended learning class and how it 
impacted students’ learning of and attitude towards statistics. 

ALEKS 

ALEKS was developed by a group of software engineers, mathematicians, and cognitive 
scientists at the University of California, Irvine and New York University with support from the 
National Science Foundation. ALEKS is an online tutoring system that takes an artificial 
intelligence approach to teaching statistics and assessing student learning individually and 
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continuously. It is commercial software that is marketed to K-12 schools, higher education, and 
private users such as parents or tutors. 

ALEKS is based upon a field of study called "Knowledge Space Theory" (Falmagne, Koppen, 
Villano, Doignon & Johannesen, 1990; Doignon & Falmagne, 1999). Knowledge Space Theory 
is a mathematical language that describes the ways in which particular elements of knowledge 
(concepts in Algebra, Mathematics, Accounting, and Statistics, for example) are organized to 
form distinct knowledge states. For example, Arithmetic is regarded as a domain of roughly one 
hundred basic concepts, giving rise to a structure of approximately 40,000 knowledge states 
(AFEKS, n.d.a). Computer algorithms have been developed to construct discipline-specific 
knowledge structures (known as "Knowledge Spaces") and apply them to assess knowledge 
states of individuals. 

AFEKS does not use multiple choice questions, but instead uses questions like traditional paper 
and pencil exercises. A student selects a module to learn, reads the online text as well as the 
examples, and then AFEKS provides a number of practice problems. After a number of 
modules, an assessment then tests the student's progress by asking 10 to 15 questions. Based on 
the assessment result, AFEKS develops a profile of the student’s knowledge: that is, which 
topics have been mastered and which have not. In subsequent assessments, AFEKS tests new 
knowledge but also includes questions from previous modules. If the student fails to show 
mastery of certain modules, he/she has to repeat them. Therefore, each set of questions is unique, 
making it impossible to predict which questions will be asked. The student can track his/her 
learning of the course topics throughout the semester, and so can the instructor. 

As for evidence of its effectiveness, some studies indicate that AFEKS improves student 
learning. In a 2002 study of 256 students in college algebra, Briggs (2007) found that students in 
three of the four sections of a course using AFEKS outperformed the control groups; 
furthermore, adopting AFEKS doubled the number of students successfully completing algebra. 
In a test of AFEKS in behavioral statistics courses (Hu, Fuellen, Okwumabua, Xu & Mo, 2007), 
the performance of African-American students in AFEKS sections closed the gap in performance 
with Caucasian students which had been in effect in earlier, lecture-based courses, and 
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essentially equalized statistics perfonnance for both groups. The ALEKS website claims an 
approximate 90% average learning rate. The website also provides testimonials from users, 
including claims that the percentage of developmental math students moving on to College 
Algebra increased from 35% to 61% with ALEKS, and that ALEKS classes had a 35% 
improvement in the number of students passing final exams in financial accounting (ALEKS, 
n.d.b). Clearly, there is a need for additional research into this product. 

Research Questions 

This study was designed to address five questions: 

1. Are there differences in performance between the blended class and a traditional offering 
of Stat I? 

2. For students in the blended class, are there any relationships between aspects of blended 
learning and performance? 

3. What advantages or disadvantages do students perceive to teaching statistics via ALEKS 
or by blending Stat I? 

4 . Is there a relationship between student perceptions of online components, ALEKS, and 
their performance? 

5. How has the experience with this blended course influenced students’ perception of 
online learning? 

The unique contribution of this study is its qualitative examination of how blended learning 
impacts student performance and attitude toward blending as well as its exploration of how 
changing the format of a class can meet students’ learning needs. 


Methodology 


Blended Stat I 

The Stat I course was blended in several ways. First, the ALEKS software was integrated into 
the Stat I course curriculum. Students were assigned modules in ALEKS corresponding to the 
topics covered in class each week and were expected to complete the modules before coming to 
class. This provided an online component that would allow students to take the time needed to 
leam new concepts, have their learning assessed, and to do so with the infinite patience and 24x7 
capabilities of a web-based program. 
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Second, class attendance was required for only five classes spread across the beginning, middle, 
and end of the course; attendance at other classes was made optional. If a student felt confident 
he/she understood the content within the module and text, they could choose not to attend class. 
However, the instructor was available during traditional class times to discuss the concepts, 
answer questions, and demonstrate problems. This structure was thought to provide students 
more flexibility in their learning choices - they could opt for online or traditional instruction - 
and to do so in a pedagogically sound manner that recognized differences in students’ 
mathematical preparation, aptitude for statistics, and learning styles. 


Research Method 

This research was conducted using a graduate-level introductory statistics course in the college 
of education of a mid-south urban university. The college of education enrolls the greatest 
percentage of graduate students at the university, totaling 1024 students or 24.6% of total 
graduate enrollment at the university in Fall 2006. Graduate students in the college are racially 
diverse as were the students included in this study (see Table 1 for the distribution by gender and 
ethnicity of the college and the sample). 


TABLE 1: Demographic Distribution of Graduate Students in College of Education (Fall 
2006) 


College of Education (Fall 2006) 


Ethnicity 

Female 

Male 

Total 

Female 

Male 

Total 

Asian 

7 

0.7% 

0 

0.0% 

7 

0.7% 

2 

2.3% 

1 

1.2% 

3 

3.5% 

African 

American 

296 

28.9% 

79 

7.7% 

375 

36.6% 

15 

17.4% 

7 

8.1% 

22 

25.6% 

Hispanic 

4 

0.4% 

3 

0.3% 

7 

0.7% 







White 

442 

43.2% 

158 

15.4% 

600 

58.6% 

38 

44.2% 

18 

20.9% 

56 

65.1% 

American 

Indian 

1 

0.1% 

0 

0.0% 

1 

0.1% 







Foreign 

19 

1.9% 

15 

1.5% 

34 

3.3% 

2 

2.3% 

3 

3.5% 

5 

5.8% 

Total 

769 

75.1% 

255 

24.9% 

1024 

100.0% 

57 

66.3% 

29 

33.7% 

86 

100.0% 


Sample (Stat I in Fall 05 & 06) 
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These students are pursuing master’s or doctoral degrees in teaching, instruction and curriculum, 
instructional design and technology, reading, early childhood education, or special education, 
human movement science and clinical nutrition, educational psychology and research, 
counseling psychology, school administration and supervision, leadership and policy studies, and 
higher and adult education. Stat I is offered every semester, often in multiple sections, and has 
only one prerequisite course, a master’s level course titled, “Introduction to Educational 
Research.” 

The blended version of Stat I was taught Fall 2006 and included 41 students in two sections. A 
traditional version of Stat I offered Fall 2005 that included 45 students was chosen for this study; 
this offering of Stat I was chosen as the comparison because students from the two semesters 
were alike in terms of gender (x 2 = 2.10, p = 0.15) and race (x 2= l-64, p =0.44). They also shared 
similar attitudes, because when asked for their reason for taking the course at the beginning of 
the course, all students responded “it is required” in both semesters. The instructor for both 
courses was the same as were all exams. Approval was gained from the institution’s Institutional 
Review Board for this study. Students signed informed consent forms if they agreed to 
participate in the study. 

Data Collection and Analysis 

Data were collected at three points in the course; these will be described in reverse chronological 
order. First, student performance was calculated by summing scores on the first and second mid- 
terms and the final exam; this composite score was used as the dependent variable measuring 
student performance. Because the exams for both offerings of Stat I (in Fall 2005 and Fall 2006) 
were exactly the same, the results were considered comparable. Demographic information about 
students (gender, ethnicity, and age) from both courses was obtained from the Institutional 
Research Office as well as the students’ incoming GRE-quantitative scores. These data were 
analyzed using ANCOVA to answer research question 1. Because of the relatively small size of 
the sample, a = 0.05 was chosen as the level of significance for hypothesis tests. 

Second, 28 students in the Fall 2006 blended class completed a three-page survey immediately 
before the final exam that included several Likert questions about blended learning, coded 


The Journal of Educators Online, Volume 5, Number 2, July 2008 


8 


“strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree.” The seven questions 
were: (1) “After learning the required topics in ALEKS, it becomes easier to read and understand 
the chapters in the textbook,” (2) “I wish ALEKS matched better with the book chapters,” (3) “I 
attend optional classes,” (4) “ALEKS helps my understanding of the lectures in the required- 
attendance classes,” (5) “I really think I would do better in this class if it were taught in 
traditional classroom format,” (6) “Overall, I enjoy online learning,” and (7) “I thi nk the class 
format is working, but changes are needed to make the ALEKS program a better fit.” 
Participation in the survey was optional and survey results were sealed and given to the 
instructor only after the course was finished and the final course grade was assigned. 

The relationship of these Likert items to the students’ performance in the class was evaluated 
using Pearson’s correlation and used to answer research question 2. 

Third, right before the end of the course, a qualitative researcher met with three focus groups to 
elicit expanded comments on the blended learning format and ALEKS. The three groups were 
defined by their cumulative scores on the first and second mid-terms, thereby forming low-, 
medium-, and high-performing groups. These sessions were video recorded and then transcribed 
for analysis. Analysis followed standard procedures for qualitative research (Maxwell, 1996), 
including the identification of themes and then a determination of whether themes were 
consistent or inconsistent across individuals or groups. Four students from each performance 
group were invited to these focus groups, but only two or three students volunteered and 
participated in each session. Their comments helped answer research questions 3 and 4. 

Finally, the three-page survey mentioned earlier also included three open-ended questions; 
answers to one question (“How did the class format influence your attitude toward online 
learning?”) were analyzed for consistency in themes and used to answer research question 5. 

Limitations 

This study was limited in three ways. First, it included only 41 students in the blended learning 
portion of the analysis. The sample is relatively small, but qualitative and quantitative findings 
were cross-validated to improve the validity of this study. Second, it represented the instructor’s 
first exploration into blended learning, and while much was learned during the process and many 
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aspects of the blended course will be changed in future offerings, it would be inappropriate to 
assume this situation would be replicated by others. Therefore, generalizing results from this 
study should be done with caution. Last, several students were allowed to discontinue working 
with ALEKS after the second mid-term exam (or approximately two-thirds into the course) 
because of their difficulty to move forward with an overwhelming number of modules to be 
relearned. This change could influence the final results of the study, but was deemed important 
to be responsive to student needs at a critical time in the class. 

Results 

Differences in Performance: Blended vs. Traditional 

Research Question 1 asked, “Are there differences in performance between the blended class and 
a traditional offering of Stat I?” Table 2 presents the results from the /-test comparing student 
performance in the blended class to that of students in a traditional class. With a highest possible 
score of 160 from the combined three exams (two midterms and one final), the table indicates 
that the blended class was seven points higher than the traditional class, but this difference was 
not statistically significant. 


TABLE 2: t-Test of Student Performance, Blended versus Traditional Class 


Group Statistics 

t test 

Year N Mean Std. Dev. 

Mean p. 

t 95CI 

Difference (2-tailed) 

Fall 2005 45 123.34 24.14 

-6.90 -1.30 0.20 (-17.43,3.63) 

Fall 2006 41 130.24 24.96 


In order to control potentially confounding variables, student's gender, ethnicity (African 
American versus others), age, and incoming quantitative GRE score were incorporated into the 
analysis. An ANCOVA analysis was conducted (Table 3) in which total exam score was the 
dependent variable; independent variables included gender, ethnicity, and class type (traditional 
vs. blended), and age and GRE-Quantitative score were the covariates. Age was found non- 
significant as a covariate and removed from the ANCOVA model to save a degree of freedom. 
The rerun of ANCOVA model without age showed that gender (F =0.56, p = 0.46), race (F 
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=2.19, p = 0. 12), and class type (F =0.07, p = 0.80) were not statistically significant influences 
on student perfonnance, but the student’s incoming quantitative score on the GRE was. After 
the GRE-Quantitative score was controlled for, student performance in the two class formats 
lacked statistical difference. The underlying assumptions of ANCOVA were also checked for 
homogeneity of variance, linearity, and homogeneity of regression. Homogeneity of variance 
was confirmed ( F =0.874, p = 0.57), and GRE-Quantitative score as the covariate was linearly 
related to the dependent variable and had no interaction effects with the independent variables. 
Thus, this result was deemed valid; it captured the student’s basic ability or preparation in math, 
which in turn influenced performance in statistics. 


TABLE 3: ANCOVA of Influences on Student Performance 


Source 

Type III SS 

df 

MS 

F 

P 

Intercept 

8110.39 

1 

8110.39 

24.77 

0.00 

Gender 

182.01 

1 

182.01 

0.56 

0.46 

Race 

1436.61 

2 

718.30 

2.19 

0.12 

Class Type 

21.86 

1 

21.86 

0.07 

0.80 

GRE-Quantitative 

11957.61 

1 

11957.61 

36.52 

0.00 

Error 

23574.75 

72 

327.43 



Total 

46850.86 

~ A r s~\ 

77 





(a) R 1 = 0.498 (Adjusted R 1 = 0.456) 


Blended Learning ’s Relationship to Performance 

Research Question 2 asked, “For students in the blended class, are there any relationships 
between aspects of blended learning and performance?” Table 4 presents the Pearson 
correlations for the Likert items dealing with blended learning and student perfonnance in the 
class. None of these Pearson correlations was close to significance. The non-significance may be 
related to insufficient statistical power due to small sample size; in any case, these Pearson 
correlations must be discarded from further analysis. 
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TABLE 4: Pearson Correlation between Blended Learning Questions and Performance 


(n=28) 


Item 

Pearson 

Correlation 

P 

“After learning the required topics in ALEKS, it becomes easier to read 
and understand the chapters in the textbook.” 

0.093 

0.638 

“I wish ALEKS matched better with the book chapters.” 

0.092 

0.641 

“I attend optional classes.” 

0.054 

0.783 

“ALEKS helps my understanding of the lectures in the required- 
attendance classes.” 

-0.143 

0.467 

“I really think I would do better in this class if it were taught in 
traditional classroom format.” 

-0.145 

0.462 

“Overall, I enjoy online learning.” 

0.063 

0.751 

“I think the class format is working, but changes are needed to make the 
ALEKS program a better fit.” 

0.093 

0.639 


Advantages and Disadvantages to ALEKS 

Research Questions 3 and 4 were combined for this analysis. Research Question 3 asked, “What 
advantages or disadvantages do students perceive to teaching statistics via ALEKS or by 
blending Stat I?” and Research Question 4 asked, “Is there a relationship between student 
perceptions of ALEKS and their performance?” Focus group transcripts were analyzed to 
answer these questions, which resulted in three main themes — ALEKS, students, and instruction 
— and several sub-themes. However, some themes were inconsistent based on the performance 
level (low, medium, high) of the student. The following discussion treats each theme and sub- 
theme by frequency of occurrence, and notes differences based on student performance. 

ALEKS . Most comments were about ALEKS and included five sub-themes: assessments, 
alignment, time, repetition, and working the system. The assessments were criticized for being 
too many and too frequent, adding topics on missed questions, and not specifying what was 
missed or misunderstood. A comment from a medium-performing student stated the problem this 
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way: “At one point I was down to like 3 or 4 [topics]. Then I had an assessment. It jumped up 
to like 27 [topics].” This student’s complaint (echoed by others) was that topics already learned 
were included in assessments, and this was both unexpected and unwelcome. The usual and 
time-honored approach to education is to deal with a topic, leam it, and go forward, only to 
return to the topic on a mid-term or final exam. ALEKS’ recursive approach to testing was not a 
boon for some students. In fact, the word most commonly applied to the assessments was 
“frustrating.” The second complaint arose when ALEKS warned the student “if you miss one 
more we’ll try something else.” The students in the group joked that ALEKS was telling the 
student “you’re a moron.” Students who are insecure may require more emotional support during 
learning, something that may be difficult for a computer program to do. 

A second sub-theme dealt with alignment of the textbook used in the course and ALEKS. Almost 
all of the participants in the focus groups commented on this issue, but the groups had different 
perceptions. The low- and medium-performing groups were more likely to stress misalignment 
of the text and ALEKS. This comment was illustrative: “The book and ALEKS. . .use different 
terms for things.” However, an individual in the high-performing group had a different view: 
“ALEKS and then the textbook really worked well together. . .reinforced the concepts.” Perhaps 
the students view alignment differently: one group focusing on an exact match of terms and 
examples, the other group focusing on how the two forms can be mutually reinforcing or 
beneficial. This interpretation requires further investigation. 

A third sub-theme was about time. Although warned by the instructor at the beginning of the 
course to study approximately 9 hours per week outside of class, several students in the low- and 
medium-performing groups mentioned the amount of time it took to work in ALEKS, especially 
the assessments. The more difficulty they had learning the material, the longer it took, and this 
was perceived to be an excessive demand on the students’ time. 

A fourth sub-theme was the role of repetition. As one individual in the low-performing group 
said, “The nice thing about [ALEKS] was you did repetitiously go through it. And so you saw it 
and saw it and saw it.” This individual perhaps understood the importance of repetition in the 
abstract, but not when facing an assessment that might take several hours to finish. 
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A fifth sub-theme was how some students worked the system. They would print off a lesson in 
ALEKS, and then log off to read it. By logging off, they would lessen the time online, delaying 
the assessment that would “pop up” every 10 hours. This should not surprise us, because if there 
are ways to work the system, students will find them. 

Students. The students commented on their own role as well. Two sub-themes arose: attitudes 
and math skills. For the sub-theme of attitudes, they stressed a student’s responsibility to come to 
class, to come prepared, and to bring textbooks to class. One student “fell behind in ALEKS,” 
which was stressful but also indicated a lack of responsibility for completing weekly 
assignments. Another attitude mentioned was fear. A student in the high-performing group 
stated, “I think a lot of people are just afraid of statistics.” Another attitude that was found 
valuable for one student was looking at math as a “puzzle-solving exercise.” 

Math skills were deemed important by many students, irrespective of performance group. But as 
one low-performing student put it: “My worst subject is math. Math has always been a challenge 
to me.” Generally, the students who had taken higher-level math courses and/or an 
undergraduate statistics course felt better about taking statistics at the graduate level. “I had to 
have an undergraduate stats class to take a graduate research methods class and it was very 
similar to the stuff that we’re learning here.” There was also some evidence of a particular 
barrier to understanding math, i.e., trouble understanding formulas. “I like ALEKS more. . .the 
book is more formulas and math and ALEKS has examples. . .if I see the examples without the 
formula. . .1 got it. But if I see the formula. . .it confuses me.” These comments may be 
particularly helpful in developing specific curricular changes that could solve these problems. 

Instruction. The students in the focus groups were also asked about the instructor and the 
instructional techniques chosen for the class. Two sub-themes emerged that may be particularly 
relevant for blended statistics: modeling and attendance. Modeling referred to two interrelated 
instructional tactics: working problems out on the board and showing the work. “I definitely get 
a lot more out of seeing somebody. . .work it out on the board and. . .explaining as they go.” One 
of the criticisms of ALEKS was “there’s no way to show your work. . . So if I’m feeding it the 
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right answer then it assumes I’m doing all the steps.” By attending class and watching the 
instructor either show the steps or critique the steps taken by the students, students saw how 
problems could be done correctly and have their missteps and misunderstandings corrected. 

Attendance, especially at the optional classes, was a more complicated topic. Students at all 
performance levels liked having optional classes. However, attending the optional classes 
became more important to the students in the low- and medium-performing groups. What was 
interesting about this was not that those who were struggling opted to attend these classes, but 
some of the reasons mentioned for why attendance mattered. Certainly, seeing problems worked 
out on the board was important, as has already been mentioned. But it was also important 
because the other students asked questions that had not yet occurred to the student: “When other 
students ask questions, then I’m like, ‘I could’ve asked that.’ But then since they asked it, [the 
instructor] answered it and I’m writing it down.” Another student added, “[hearing the] dialogue 
back and forth. . .then getting to actually ask a question the way you need to. . .you can’t do that 
on the computer.” In other words, it took attending class with other students and a good 
instructor and watching the interaction among them to ask a really good question. 

These themes and sub-themes indicate that students found advantages and disadvantages to 
blending using ALEKS. There is consistent evidence that students in the high-performing group 
saw the experience differently from peers in the low- and medium-performing groups. 
Performance levels made the experiences of students different, from frustration to appreciation 
and struggle to ease. 

Influence on Perceptions 

Research Question 5 asked, “Has the experience with this blended course influenced 
students’ perception of online learning?” The results are interesting and can be summarized as “I 
like online, but. . .” This interesting twist became apparent when the responses to the open-ended 
question on the survey were incorporated into the analysis. The question asked was, “How did 
the class fonnat influence your attitude toward online learning?” As phrased, the question was 
not a direct assessment of ALEKS, but answers would clearly be influenced by feelings 
generated by the ALEKS experience. 
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Of the 28 students responding to this open-ended question, eight expressed negative opinions 
about online learning and two expressed no opinion, but the remainder - 18 students - expressed 
positive opinions. One might think this result odd in comparison to the often intense negative 
comments garnered during the focus groups, but in fact, many of the comments that were clearly 
supportive of online learning contained a “but.” The “buts” sound familiar, however. They liked 
online learning, but for the “assessments” (four mentions), a lack of alignment between the text 
and ALEKS (four mentions), too much repetition (one mention), and a lack of Internet access at 
home (one mention). The relative consistency with the results from the focus groups tended to 
confirm that despite low numbers of participants in the focus groups, the analysis of the groups’ 
comments captured the issues that concerned all of the students in the blended class. In 
conclusion, despite some difficulties with ALEKS, these students liked online learning. This is 
an interesting endorsement from experienced — and critical — consumers. 

Discussion and Recommendations 

This research represents the first time the instructor attempted to teach Stat I in a blended 
learning context and to evaluate this teaching approach both quantitatively and qualitatively. 
Much was learned that can guide the revision of the course and provide better understanding of 
how the format of a class influences student learning. First and most clearly, math skills and 
preparation for statistics seemed to make a difference in performance, as evidenced by the 
comments in the focus groups and the influence on student performance of the students’ 
incoming GRE-Quantitative scores. This argues for the need for some preparatory units on 
essential math skills or mathematical concepts or better attention to the GRE-Quantitative score 
when admitting new graduate students. 

Second, in the blended class, the online component was clearly important. While most students 
had a positive attitude towards online learning and the flexibility of blended learning, many 
suffered setbacks because they did not appreciate the learning theory behind ALEKS, including 
its emphasis on mastery learning which required the repetition of earlier concepts. Also, based 
on student comments, there needs to be a good alignment of the text and online system, including 
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direct connections between approaches and language. The consistency between the two 
components - face-to-face and online — of blended courses warrants more attention as more 
classes experiment with various commercial software packages such as ALEKS. 

ALEKS and similar commercial software learning systems may also need to consider adjusting 
their approach. Students may need better explanations of what was missed in an assessment as 
well as less frequent assessments with fewer items. At least for students who are already 
struggling with learning statistics, they might do better with more regular but smaller, discrete 
assessments. And it may be worthwhile for commercial vendors to align their software to work 
closely with an existing textbook that is widely used, since this seemed to be a problem for many 
of the Stat I students. 

What is there to make of the lack of relationship between the Likert items on various aspects of 
blended learning and student performance? Certainly any conclusions must be tentative as there 
were few (n=28) responses to the survey, and this might affect the reliability of the results. But it 
is likely that there was no relationship, and that blending as was done in this class had little effect 
on performance. Put another way, perhaps quantitative ability and skill as captured in the GRE- 
Quantitative score is more critical to learning a subject like statistics than experimenting with the 
instructional format. That is an assertion that should clearly be tested in a future study. On the 
other hand, given the systematic variations that emerged from the students at different 
performance levels in regards to blended learning and ALEKS, future research ought to explore 
how to modify the elements of blended learning and adjust instructional approaches to meet the 
needs of students at different levels. 

The larger issue that remains open is whether uses of software such as ALEKS can help improve 
the understanding of statistics for students who may be math-averse or do not possess strong 
preparation in math. This situation is common in education, as was noted in the introduction, 
because the majority of students specialize in instruction, psychology, or their primary field of 
study. Since students with problematic attitudes toward math or inadequate quantitative skills 
comprise a portion of the students in many fields, how can blending be used strategically to 
address holes in the preparation or understanding of students? Certainly, merely adding online 
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components to make a blended course may not improve student learning, unless it is done in a 
thoughtful and strategic manner that addresses specific student learning needs. Can blending do 
this so that students gain confidence rather than question their abilities or feel like a 
moron? If blending can address this problem, then it will have proven its worth to students 
wherever and whatever they study. 


The Journal of Educators Online, Volume 5, Number 2, July 2008 


18 


References 

ALEKS. (n.d.a). Overview of ALEKS. Retrieved June 6, 2007 from 
http://www.aleks.com/about aleks/overview . 

ALEKS. (n.d.b) Success stories. Retrieved June 6, 2007 from 
http://www.aleks.com/highered/success stories 

Bonk, C. J., & Graham, C. R. (2005). The handbook of blended learning: Global perspectives, 
local designs. New York: Pfeiffer. 

Briggs, L.L. (2007, April 25). Software boosts math success rates at Black Hills State U. 
Campus Technology. Retrieved June 6, 2007 from 
http://www.campustechnologv.com/article.aspx?aid=476 1 5 

Chen, C. C., & Jones, K. T. (2007). Blended learning vs. traditional classroom settings: 
Assessing effectiveness and student perceptions in an MBA accounting course. The 
Journal of Educators Online, 4(1). Retrieved October 23, 2007 from 
http://www.theieo.com/Volume4Numberl/JonesFinal.pdf 

Doignon, J., & Falmagne, J. (1999). Knowledge Spaces. Berlin: Springer-Verlag. 

Dziuban, C., Hartman, J., Moskal, P., Sorg, S., & Truman, B. (2004). Three ALN modalities: 
An institutional perspective. In J. Bourne & J.C. Moore (Eds.), Elements of Quality Online 
Education: Into the Mainstream (pp. 127-148). Needham, MA: Sloan-C Consortium. 

Falmagne, J., Koppen, M., Villano, M., Doignon, J., & Johannesen, L. (1990). Introduction to 
knowledge spaces: How to build, test, and search them. Psychological Review, 97, 201- 
224. 

Graham, C. R., & Robison, R. (2007). Realizing the transformational potential of blended 
learning. In Picciano, A. G., and Dziuban, C. D. (Eds.), Blended learning: Research 
perspectives (pp. 83-1 10). Needham, MA: Sloan Consortium. 

Hu, X., Luellen, J.K., Okwumabua, Xu, Y., & Mo, L. (2007). Observational findings from a 

web-based intelligent tutoring system: Elimination of racial disparities in an undergraduate 
behavioral statistics course. Paper presented at the 2007 Annual Meeting of the American 
Educational Research Association (AERA). Chicago, IL: April 2007. 

Kaleta, R., Skibba, K., Joosten, T. (2007). Discovering, designing, and delivering hybrid 
courses. In Picciano, A. G., and Dziuban, C. D. (Eds.), Blended learning: Research 
perspectives (pp. 1 1 1-144). Needham, MA: Sloan Consortium. 


The Journal of Educators Online, Volume 5, Number 2, July 2008 


19 


Maxwell, J. A. (1996). Qualitative research design: An interactive approach. Thousand Oaks, 
CA: Sage Publications. 

McCombs, B. & Vakili, D. (2005). A learner-centered framework for e-learning. Teachers 
College Record, 107, 1582-1500. 

Meyer, K.A. (2004, Spring). Putting the distance learning comparison study in perspective: Its 
role as personal journey research. The Online Journal of Distance Learning 
Administration, 7(1). Retrieved June 6, 2007 from 
http://www.westga.edu/~distance/ojdla/spring7 l/meyer7 1 .pdf 

Picciano, A. G. (2006). Blended learning: Implications for Growth and Access. Journal of 
Asynchronous Learning Networks, 10(3), 95-102. 

Picciano, A. G., & Dziuban, C. D. (Eds.) (2007). Blended learning: Research perspectives. 
Needham, MA: Sloan Consortium. 

Reasons, S., Valadares, K., & Slavkin, M. (2005). Questioning the hybrid model: Student 

outcomes in different course formats. Journal of Asynchronous Learning Networks, 9(1), 
83-94. 

Russell, T.L. (1999). The No Significance Phenomenon. Raleigh, NC: North Carolina State 
University. 

Thorne, K. (2003). Blended learning: How to integrate online and traditional learning. 
Sterling, VA: Kogan Page. 

Vaughan, N. (2007). Perspectives on blended learning in higher education. International 
Journal on E-Learning, (5(1) 81-94. 

Vignare, K. (2007). Review of the literature on blended learning. In Picciano, A. G., and 

Dziuban, C. D. (Eds.), Blended learning: Research perspectives (pp. 37-64). Needham, 
MA: Sloan Consortium. 

Wu, D., & Hiltz, S. R. (2004). Predicting learning from asynchronous online discussions. 
Journal of Asynchronous Learning Networks, 8(2), 139-151. 


The Journal of Educators Online, Volume 5, Number 2, July 2008 


20 


